Inspecting for a defect on a print medium with an image aligned based on an object in the image and based on vertices of the inspection target medium and the reference medium

ABSTRACT

There is provided with an image processing apparatus. An obtaining unit obtains a first image serving as a read image of an inspection target medium having undergone printing, and a second image serving as a read image of a reference medium representing a target print result. An inspection unit inspects a defect on the inspection target medium based on the first image and the second image by performing inspection at inspection settings different between a print region and a peripheral region of the inspection target medium.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a technique of detecting a defect on aprinted product.

Description of the Related Art

A printed product output from a printing apparatus sometimes has a stainarising from attachment of a color material to an unintended portion, ora color loss arising from a failure of attachment of a sufficient amountof color material to a portion where an image should be formed. Thequality of a printed product needs to be guaranteed by inspecting thepresence/absence of such print defects. A visual inspection by aninspector requires a long time and high cost, so inspection systems thatautomatically perform inspection are proposed recently. Such aninspection system can detect the presence/absence of a print defect onan inspection target based on a difference between a reference imageserving as image data representing a defect-free printed product, and atarget image serving as image data of a printed product serving as theinspection target.

In general, the positional relationship between a medium and a pictureprinted on the medium slightly varies every printing owing to variationsof a print position along with an error of a medium conveyance positionin a printing apparatus, variations of the size of a medium along withan error of a cutting position, or the like. When automaticallyinspecting a printed product, alignment needs to be performed to makethe pictures of a reference image and a target image match with eachother. For example, Japanese Patent Laid-Open No. 2015-065647 disclosesan apparatus that inspects a printed product by comparing a read imageof a printed product with a master image. In Japanese Patent Laid-OpenNo. 2015-065647, a master image is obtained by aligning each of a readimage of preprint paper before printing and a print image to be printedon the preprint paper with a read image of a printed product based onreference points such as edge points, and then compositing them.

SUMMARY OF THE INVENTION

According to an embodiment of the present invention, an image processingapparatus comprises: an obtaining unit configured to obtain a firstimage serving as a read image of an inspection target medium havingundergone printing, and a second image serving as a read image of areference medium representing a target print result; and an inspectionunit configured to inspect a defect on the inspection target mediumbased on the first image and the second image by performing inspectionat inspection settings different between a print region and a peripheralregion of the inspection target medium.

According to another embodiment of the present invention, an imageprocessing apparatus comprises: an obtaining unit configured to obtain afirst image serving as a read image of a medium having undergoneprinting, and a second image different from the first image; and acompositing unit configured to composite the first image and the secondimage by performing compositing processing at compositing settingsdifferent between a print region and peripheral region of the medium.

According to still another embodiment of the present invention, an imageprocessing method comprises: obtaining a first image serving as a readimage of an inspection target medium having undergone printing, and asecond image serving as a read image of a reference medium representinga target print result; and inspecting a defect on the inspection targetmedium based on the first image and the second image by performinginspection at inspection settings different between a print region and aperipheral region of the inspection target medium.

According to still another embodiment of the present invention, an imageprocessing method comprises: obtaining a first image serving as a readimage of a medium having undergone printing, and a second imagedifferent from the first image; and compositing the first image and thesecond image by performing compositing processing at compositingsettings different between a print region and peripheral region of themedium.

According to yet another embodiment of the present invention, anon-transitory computer-readable medium stores a program which, whenexecuted by a computer, causes the computer to perform a methodcomprising: obtaining a first image serving as a read image of aninspection target medium having undergone printing, and a second imageserving as a read image of a reference medium representing a targetprint result; and inspecting a defect on the inspection target mediumbased on the first image and the second image by performing inspectionat inspection settings different between a print region and a peripheralregion of the inspection target medium.

According to yet another embodiment of the present invention, anon-transitory computer-readable medium stores a program which, whenexecuted by a computer, causes the computer to perform a methodcomprising: obtaining a first image serving as a read image of a mediumhaving undergone printing, and a second image different from the firstimage; and compositing the first image and the second image byperforming compositing processing at compositing settings differentbetween a print region and peripheral region of the medium.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view showing the arrangement of a printing system;

FIG. 2 is a block diagram showing the functional arrangement of an imageprocessing apparatus;

FIG. 3 is a flowchart showing a sequence of an image processing method;

FIGS. 4A and 4B are views exemplifying a picture region 305 and aperipheral region 306;

FIGS. 5A to 5C are views for explaining an outline of alignmentprocessing;

FIG. 6 is a block diagram showing the functional arrangement of an imageprocessing apparatus;

FIG. 7 is a flowchart showing a sequence of an image processing method;

FIGS. 8A to 8C are views for explaining an outline of alignmentprocessing;

FIGS. 9A to 9C are views for explaining an overall transformed image;

FIGS. 10A and 10B are views for explaining defect detection at theperiphery of a print medium;

FIG. 11 is a flowchart showing a sequence of processing of generatingthe peripheral region of a reference image;

FIG. 12 is a flowchart showing a sequence of inspection processing forthe peripheral region; and

FIG. 13 is a flowchart showing a sequence of inspection processingincluding inspection of a preprint region.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments will be described in detail with reference tothe attached drawings. Note that the following embodiments are notintended to limit the scope of the claimed invention, and limitation isnot made an invention that requires all combinations of featuresdescribed in the embodiments. Two or more of the multiple featuresdescribed in the embodiments may be combined as appropriate.Furthermore, the same reference numerals are given to the same orsimilar configurations, and redundant description thereof is omitted.

In Japanese Patent Laid-Open No. 2015-065647, a master image based onprint image data used for printing and a read image of a printed productare compared. To guarantee the quality of an actual printed product, aread image of a defect-free printed product is desirably used as areference image. However, when the positional relationship between amedium and a picture printed on the medium varies, as described above,if alignment is performed to make printed pictures in a target image anda reference image match with each other, the position of the peripheryof the medium may change to increase a difference near the periphery ofthe medium. As a result, an inaccurate inspection result may be obtainedat the periphery of the medium.

An embodiment of the present invention enables a proper inspection of aprinted product even when a print position error occurs.

First Embodiment

An image processing apparatus according to the first embodiment inspectsthe presence/absence of a defect on an inspection target medium havingundergone printing. In this embodiment, out of the inspection targetmedium, a region where printing is performed (region where a colormaterial is fixed and a picture is formed; to be simply referred to as apicture region), and a region at the periphery of the medium where noprinting is performed (to be simply referred to as an peripheral region)are determined, and inspection processing corresponding to each regionis selected. According to the embodiment, even when a print positionerror occurs, a defect at the periphery of a medium can be detected moreaccurately.

(Arrangement of Printing System)

FIG. 1 is a view showing the arrangement of a whole printing system thatincludes an image processing apparatus 100 serving as an imageprocessing apparatus according to the first embodiment, and performsoutput and inspection of a printed product. The printing systemaccording to the first embodiment includes the image processingapparatus 100 and a printing apparatus 190. The printing systemaccording to the first embodiment may further include a printing server180.

The printing server 180 generates a print job including an original tobe printed, and inputs the print job to the printing apparatus 190. Theprinting apparatus 190 forms an image on a print medium based on theprint job input from the printing server 180. The printing apparatus 190includes a sheet feeding unit 191, and the user can supply in advanceprint sheets to the sheet feeding unit 191. When the print job is input,the printing apparatus 190 forms an image on one or two surfaces of aprint medium supplied to the sheet feeding unit 191 while conveying theprint medium along a conveyance path 192. Then, the printing apparatus190 sends the print medium to the image processing apparatus 100.

The image processing apparatus 100 inspects a defect on a print medium(inspection target medium) having undergone printing. The inspectiontarget medium is obtained by forming an image on the print medium by theprinting apparatus 190, and is conveyed through the conveyance path 192inside the printing apparatus 190. The image processing apparatus 100may incorporate a CPU 101, a RAM 102, and a ROM 103. The imageprocessing apparatus 100 may include an image reading device 105, aprinting apparatus interface (I/F) 106, a general-purpose interface(I/F), a user interface (UI) panel 108, and a main bus 109. Further, theimage processing apparatus 100 may have a conveyance path 110 for aprint medium that is connected to the conveyance path 192 of theprinting apparatus 190. The image processing apparatus 100 may includean output tray 111 to which an inspection target medium determined to beaccepted by inspection is output, and an output tray 112 to which aninspection target medium determined to be rejected by inspection isoutput. In the example of FIG. 1 , the output trays 111 and 112 areconnected to the CPU 101 via the main bus 109. The output tray 111 or112 is set as the conveyance destination of an inspection target mediumin accordance with the result of inspection on the inspection targetmedium.

The image processing apparatus according to each embodiment to bedescribed later can be implemented by a computer including a processorand a memory. For example, the function of each unit shown in FIGS. 2, 6, and the like to be described later can be implemented by executing aprogram stored in a memory such as the RAM 102 or the ROM 103 by aprocessor such as the CPU 101. The processor such as the CPU 101 canalso control each module in the image processing apparatus 100, asneeded. Note that the image processing apparatus according to theembodiment of the present invention may be constituted by, for example,a plurality of processing apparatuses connected via a network.

The CPU 101 is a processor that controls each unit in the imageprocessing apparatus 100. The RAM 102 temporarily holds an applicationexecuted by the CPU 101, data used for image processing, or the like.The ROM 103 stores programs executed by the CPU 101.

The image reading device 105 reads, on the conveyance path 110, one ortwo surfaces of a print medium sent from the printing apparatus 190, andobtains them as image data. Since the conveyance path 110 serves as abackground when the image reading device 105 reads an image on a printmedium, it can be constituted to have a color (for example, black)easily discriminable from a print medium on an image. The printingapparatus I/F 106 is connected to the printing apparatus 190, and theimage processing apparatus 100 can communicate with the printingapparatus 190 via the printing apparatus I/F 106. For example, theprinting apparatus 190 and the image processing apparatus 100 can besynchronized via the printing apparatus I/F 106 to notify each other oftheir operating situations. The UI panel 108 can output information tothe user. The UI panel 108 may be a display device such as a liquidcrystal display and can function as the user interface of the imageprocessing apparatus 100. The UI panel 108 can notify the user of, forexample, the current situation or settings of the image processingapparatus 100. The UI panel 108 may include an input device such as atouch panel or a button and can accept an instruction from the user. Themain bus 109 is a transmission path that connects the modules of theimage processing apparatus 100.

The image processing apparatus 100 performs inspection processing tocheck the presence/absence of a defect on a print medium based on imagedata of the print medium obtained by the image reading device 105 whilethe print medium output from the printing apparatus 190 is conveyedalong the conveyance path 110. If the print medium is determined to beaccepted as a result of the inspection processing, it is conveyed to theoutput tray 111. If the print medium is determined to be rejected as aresult of the inspection processing, it is conveyed to the output tray112. By this operation, only print media determined to have no defectare output onto the output tray 111.

(Functional Arrangement of Image Processing Apparatus)

FIG. 2 is a block diagram showing the functional arrangement of theimage processing apparatus 100 according to the embodiment. Eachrectangular frame represents a functional module that performs eachprocessing according to the embodiment, and each arrow indicates a flowof data. The arrangement in FIG. 2 is merely an example, and the imageprocessing apparatus 100 according to the embodiment is not limited toone shown in FIG. 2 .

An image obtaining unit 201 obtains the first image serving as a readimage of an inspection target medium having undergone printing, and thesecond image serving as a read image of a reference medium representinga target print result. In this embodiment, the image obtaining unit 201obtains image data obtained by reading a print medium on the conveyancepath 110 by the image reading device 105. The image reading device 105reads print media at least at two different times and generatesrespective image data of the print media. An image obtained by reading areference medium at the first time by the image reading device 105 willbe called a reference image 301, and an image obtained by reading aninspection target medium at the second or subsequent time will be calleda target image 302.

A region determination unit 202 determines a picture region 305 and aperipheral region 306 in each of the reference image 301 and the targetimage 302 obtained by the image obtaining unit 201. The regiondetermination unit 202 can determine a region to which each pixel ofimage data belongs. In this embodiment, the region determination unit202 divides the target image 302 into the picture region 305 and theperipheral region 306. The region determination unit 202 can also dividethe reference image 301 into the picture region 305 and the peripheralregion 306.

A processing selection unit 203 sets an inspection to be performed by aninspection processing unit 205 in accordance with the type of a regiondetermined by the region determination unit 202. In this embodiment, theprocessing selection unit 203 selects inspection processing used foreach of the picture region 305 and the peripheral region 306.

An alignment unit 204 aligns the reference image 301 and the targetimage 302. The alignment unit 204 can align images in accordance withthe brightness of the images. For example, the alignment unit 204 canalign the reference image 301 and the target image 302 based on objectsin the images. More specifically, the alignment unit 204 can calculate ageometric correction parameter so that when geometric correction isperformed on one image, an object in one image comes close to an objectin the other image.

The alignment unit 204 according to the embodiment performs alignmentwith the reference image 301 for the picture region 305 of the targetimage 302. For example, the alignment unit 204 can calculate a positiondisplacement amount between images in the picture regions 305 of thetarget image 302 and reference image 301 based on the brightness of thepicture region 305 observed from each of the target image 302 and thereference image 301. The position displacement amount calculated in thisembodiment can include a translation amount, a rotation amount, and ascaling amount.

The inspection processing unit 205 inspects a defect on an inspectiontarget medium based on the reference image 301 and the target image 302.In this embodiment, the inspection processing unit 205 performsinspection at inspection settings different between the print region andperipheral region of an inspection target medium. The processingselection unit 203 makes inspection settings. In this embodiment, as forthe print region (that is, the picture region 305 of the target image302) of the inspection target medium, the inspection processing unit 205determines a defect based on a comparison between the target image 302and the reference image 301. For example, the inspection processing unit205 can calculate a difference between the reference image 301 and thetarget image 302 and perform inspection in a state in which thealignment unit 204 aligns them. The inspection processing unit 205 candetermine that a portion having a large difference between the referenceimage 301 and the target image 302 is a defective portion. As for theperipheral region (that is, the peripheral region 306 of the targetimage 302) of the inspection target medium, the inspection processingunit 205 determines a defect at an inspection setting different fromthat of the print region. A detailed method will be described later.

(Processing Executed by Image Processing Apparatus)

Processing performed by the image processing apparatus 100 having theabove-described arrangement according to the embodiment will bedescribed below. FIG. 3 is a flowchart showing a sequence of processingperformed by the image processing apparatus 100.

In step S1010, the image obtaining unit 201 obtains the reference image301 obtained by reading a reference medium on the conveyance path 110 bythe image reading device 105 in the above-described manner, and storesit in the RAM 102. The reference image 301 is used for subsequentprocessing to inspect each target image 302.

In step S1020, the image obtaining unit 201 obtains the target image 302obtained by reading an inspection target medium on the conveyance path110 by the image reading device 105 in the above-described manner, andstores it in the RAM 102. In subsequent steps S1030 to S1070, inspectionof the target image 302 obtained in step S1020 is performed.

In step S1010, the image reading device 105 can generate the referenceimage 301 by reading the reference medium. In step S1020, the imagereading device 105 can generate the target image 302 by reading theinspection target medium. The embodiment is also applicable to a case inwhich the reference image 301 or the target image 302 obtained usinganother device is used. For example, the reference image 301 or thetarget image 302 obtained by a device different from the image readingdevice 105 may be stored in an auxiliary storage device (not shown). Inthis case, the image obtaining unit 201 can obtain the reference image301 or the target image 302 from the auxiliary storage device in stepS1010 or S1020.

FIG. 4A is a view showing an example of the target image 302. The targetimage 302 includes a print medium region 304 where a print medium iscaptured, and a background region 303 where no print medium is captured.In the example of FIG. 4A, the image reading device 105 reads the targetimage 302 so that the conveyance path 110 is captured as a backgroundaround the print medium region 304 in order to obtain the image of theentire print medium. In this embodiment, the conveyance path 110 isblack, so the background region 303 becomes black. Similar to the targetimage 302, the reference image 301 includes a print medium region wherea print medium is captured, and a background region where no printmedium is captured.

In step S1030, the region determination unit 202 detects the pictureregion 305 and the peripheral region 306 in the target image 302 byperforming region determination for the target image 302. In thisexample, the region determination unit 202 first extracts the printmedium region 304 from the target image 302. The region determinationunit 202 can extract the print medium region 304 in the following way.First, the region determination unit 202 binarizes the target image 302,traces pixels serving as the boundary between white pixels and blackpixels, and linearly approximates four sides representing the outline ofthe print medium. As a result, four straight lines representing theoutline of the print medium region 304 are estimated. Then, the regiondetermination unit 202 calculates the intersection points of the fourstraight lines. The intersection points correspond to vertices 308 a to308 d of the print medium region 304 shown in FIG. 4A. Needless to say,the method of extracting the print medium region 304 is not limited tothe above-described one, and another method is also available.

After that, the region determination unit 202 detects the picture region305 and the peripheral region 306 from the print medium region 304 ofthe target image 302. In this embodiment, the region determination unit202 determines to which of the picture region 305 and the peripheralregion 306 each pixel constituting the print medium region 304 belongs.FIG. 4B shows the relationship between the target image 302, the printmedium region 304, the picture region 305 (shown in FIG. 4B as a regionwhere white and pictures exist), and the peripheral region 306 (hatchedin FIG. 4B). In the example of FIG. 4B, the inside of the print mediumregion 304 belongs to either the picture region 305 or the peripheralregion 306.

In this embodiment, the region determination unit 202 can determine, asthe peripheral region 306, a region within a predetermined distance fromthe outline of the medium on the reference image 301 or the target image302. For example, the region determination unit 202 can extract, as theperipheral region 306, a region within a predetermined distance from theoutline of the print medium region 304, and extract the remaining regionas the picture region 305. The predetermined distance that defines theperipheral region 306 can be called a peripheral margin and set inadvance by the user prior to processing.

The method of detecting the peripheral region 306 is not limited to theabove-described one. For example, the region determination unit 202 candetect a marker applied to a medium on the reference image 301 or thetarget image 302, and determine the peripheral region 306 based on thecoordinates of the marker. More specifically, when an alignment markersuch as a register mark is printed on a print medium, the regiondetermination unit 202 may set a peripheral margin based on thecoordinates of the alignment marker. The peripheral margin may be setbased on a margin amount or a print position error amount when theprinting apparatus 190 prints on a medium on the reference image 301 orthe target image 302. For example, the region determination unit 202 mayobtain information representing the average values of a margin size andprint error amount at the time of printing from the printing apparatus190, and set the sum of the average values of the margin size and printerror amount as the peripheral margin. Further, the region determinationunit 202 may determine the plain region of the print medium region 304as the peripheral region 306 and the remaining region as the pictureregion 305.

In this embodiment, the region determination unit 202 calculates featureinformation (in this example, average brightness) about the peripheralregion 306 detected from the reference image 301. The featureinformation is used in step S1060, details of which will be describedlater.

By performing similar processing on the reference image 301, the regiondetermination unit 202 can extract the print medium region 304 of thereference image 301. Also, the region determination unit 202 can extractthe picture region 305 and peripheral region 306 of the reference image301. Note that the image obtaining unit 201 may extract the print mediumregion 304, picture region 305, and peripheral region 306 of thereference image 301 in step S1010.

In step S1040, the processing selection unit 203 selects the details ofdetection processing to be performed for the picture region 305 in stepS1060, and those of inspection processing to be performed for theperipheral region 306 in step S1070. In step S1060, defect determinationis performed for the picture region 305 at the settings selected by theprocessing selection unit 203. In step S1070, defect determination isperformed for the peripheral region 306 at the settings selected by theprocessing selection unit 203. The details of inspection processingselected by the processing selection unit 203 can be set in advance bythe user prior to processing. In this embodiment, the processingselection unit 203 selects defect determination processing based on acomparison between the reference image 301 and the target image 302 forthe picture region 305, and defect determination processing based on acomparison between the target image 302 and feature information for theperipheral region 306.

In step S1050, the alignment unit 204 aligns the reference image 301 andthe target image 302. In this embodiment, defect determinationprocessing based on a comparison between the reference image 301 and thetarget image 302 is performed only for the picture region 305. Hence,the alignment unit 204 aligns the reference image 301 and the targetimage 302 based on the picture region 305.

An example of processing performed by the alignment unit 204 will beexplained with reference to FIGS. 5A to 5C showing an outline ofalignment processing. FIG. 5A shows the picture region 305 and referencepoints 307 a to 307 c in the reference image 301. FIG. 5B shows thepicture region 305 and the reference points 307 a to 307 c in the targetimage 302. The reference point 307 is a point indicating the sameportion of the same object in the picture regions 305 of the referenceimage 301 and the target image 302, and is also called a feature point.In this embodiment, the three reference points 307 a to 307 c are setand used for alignment. In FIGS. 5A and 5B, the same reference numeralsdenote corresponding reference points. For example, the reference point307 a of the reference image 301 and the reference point 307 a of thetarget image 302 indicate the same portions of a picture though thecoordinates in the respective images are different. This also applies tothe reference points 307 b and 307 c.

In this embodiment, the positions of the reference points 307 a to 307 cwith respect to a picture to be printed by the printing apparatus 190are set in advance. Note that the user may manually set the referencepoints 307 a to 307 c while checking the picture. The reference points307 a to 307 c may be automatically set using an image feature amountsuch as SIFT or SURF. The alignment unit 204 can detect the setreference points 307 a to 307 c by image analysis on the picture regions305 of the reference image 301 and the target image 302. In thisembodiment, printing is performed on an inspection target medium inaccordance with preset print data, so rough appearance positions of thereference points can be predicted. The alignment unit 204 can detect thereference points 307 a to 307 c by setting windows of a predeterminedsize (for example, 32×32 pixels) near the predicted positions of thereference points 307 a to 307 c, and performing template matching. Asanother method, the alignment unit 204 may detect the reference points307 a to 307 c using processing of detecting an image feature amountsuch as SIFT or SURF mentioned above.

The alignment unit 204 detects the reference points 307 a to 307 c,obtaining the coordinates of the respective reference points 307 a to307 c in each of the reference image 301 and the target image 302. Thecoordinates of the reference points 307 a, 307 b, and 307 c in thereference image 301 are defined as (x1 a, y1 a), (x1 b, y1 b), and (x1c, y1 c), respectively. The coordinates of the reference points 307 a,307 b, and 307 c in the target image 302 are defined as (x2 a, y2 a),(x2 b, y2 b), and (x2 c, y2 c).

Then, the alignment unit 204 aligns the reference image 301 and thetarget image 302 based on the positions of the corresponding referencepoints 307 a to 307 c detected from the reference image 301 and thetarget image 302. For example, when geometric transformation is appliedto the reference image 301, the alignment unit 204 can calculate ageometric transformation for alignment so that the positions of thereference points 307 a to 307 c in the reference image 301 and thetarget image 302 match with each other. In this embodiment, thealignment unit 204 calculates a transformation X of the picture region305 from the reference image 301 to the target image 302 according toequation (1):AX=B   (1)where

$A = \begin{pmatrix}{x1a} & {y1a} & 1 \\{x1b} & {y1b} & 1 \\{x1c} & {y1c} & 1\end{pmatrix}$ $B = \begin{pmatrix}{x2a} & {y2a} & 1 \\{x2b} & {y2b} & 1 \\{x2c} & {y2c} & 1\end{pmatrix}$ $X = \begin{pmatrix}{x11} & {x12} & {x13} \\{x21} & {x22} & {x23} \\0 & 0 & 1\end{pmatrix}$

In equation (1), X is an affine transformation and represents adisplacement amount obtained by combining translation, rotation,scaling, and shearing from the reference image 301 to the target image302. The transformation X can be calculated by multiplying the inversematrix of A from the right side of B in equation (1).

After that, the alignment unit 204 generates a transformed image 309 byapplying the calculated transformation X to each pixel constituting theprint medium region 304 in the reference image 301. As shown in FIG. 5C,the picture in the picture region 305 of the generated transformed image309 is aligned with the target image 302. That is, the positions of thereference points 307 a to 307 c in the transformed image 309 and thetarget image 302 match with each other. In this manner, the alignmentunit 204 can align the reference image 301 and the target image 302.

In step S1060, the inspection processing unit 205 performs for theperipheral region 306 inspection processing selected by the processingselection unit 203 in step S1040. As described above, in thisembodiment, the inspection processing unit 205 performs defectdetermination processing based on a comparison between the target image302 and feature information for the peripheral region 306. Morespecifically, the inspection processing unit 205 can detect a defect inthe peripheral region 306 of the inspection target medium by comparingimage information at each position of the peripheral region 306 of thetarget image 302 with feature information having a predetermined value.

The image information at each position of the peripheral region 306 ofthe target image 302 may be the brightness of each pixel. The featureinformation (to be referred to as a feature value hereinafter) having apredetermined value may be feature information about the peripheralregion 306. The feature information may be, for example, therepresentative brightness of pixels belonging to the peripheral region306. The representative brightness is a brightness representing theperipheral region 306. For example, when a print medium is white paper,the representative brightness may be the typical brightness of whitepaper read by the image obtaining unit 201.

In this embodiment, the inspection processing unit 205 uses, as thefeature value, feature information about the peripheral region 306 ofthe reference image 301 obtained by the image obtaining unit 201. Inthis embodiment, the inspection processing unit 205 uses, as the featurevalue, the average brightness of the peripheral region 306 detected fromthe reference image 301 in step S1030. Then, the inspection processingunit 205 calculates a difference between the pixel value and the featurevalue for each pixel of the peripheral region 306 of the target image302. The inspection processing unit 205 can determine that a pixelhaving a calculated difference larger than a threshold is a defectivepixel.

Note that the inspection processing unit 205 may use a preset value asthe feature value instead of the representative brightness of theperipheral region 306 detected from the reference image 301. Forexample, the feature value can be set in advance in accordance with thetype of print medium. This setting can omit the processing of detectingthe peripheral region 306 of the reference image 301 by the regiondetermination unit 202. Even when a print position error occurs, aprinted product can be inspected properly.

In step S1070, the inspection processing unit 205 performs for thepicture region 305 inspection processing selected by the processingselection unit 203 in step S1040. As described above, in thisembodiment, the inspection processing unit 205 performs defectdetermination processing based on a comparison between the referenceimage 301 and the target image 302 for the picture region 305. Morespecifically, the inspection processing unit 205 detects a defect in thepicture region 305 of the inspection target medium by comparing thetarget image 302 and reference image 301 aligned based on objects in theimages. As described above, the alignment unit 204 generates thetransformed image 309 by alignment processing of the reference image 301to the target image 302 in step S1050. The inspection processing unit205 calculates a difference between the target image 302 and thetransformed image 309 for each pixel of the picture region 305 in thetarget image 302. The inspection processing unit 205 can determine thata pixel having a calculated difference larger than a threshold is adefective pixel.

When the inspection processing unit 205 detects a defective pixel instep S1060 or S1070, the inspection target medium is determined to berejected. At this time, the CPU 101 of the image processing apparatus100 controls to convey the inspection target medium to the output tray112. When the inspection processing unit 205 detects no defective pixel,the inspection target medium is determined to be accepted. At this time,the CPU 101 of the image processing apparatus 100 controls to convey theinspection target medium to the output tray 111.

In step S1080, the inspection processing unit 205 determines whether toconvey the next inspection target medium to the image processingapparatus 100. If the printing apparatus 190 further prints on a printmedium and the inspection target medium is conveyed to the imageprocessing apparatus 100, the process returns to step S1020 to continuethe processing in FIG. 3 . If printing and inspection end, theprocessing in FIG. 3 ends.

In the above-described example, the alignment unit 204 aligns thereference image 301 and the target image 302 by transforming thereference image 301 using the transformation X in step S1050.Alternatively, the alignment unit 204 may align the target image 302with respect to the reference image 301 by transforming the target image302. For example, a transformed image can be generated by transformingthe target image 302 using a different transformation Y. Thetransformation Y can be given by equation (2):BY=A   (2)

In equation (2), the matrices A and B are the same as those in equation(1). In the transformed image, a picture in the picture region 305 isaligned with the reference image 301. In this case, the inspectionprocessing unit 205 can calculate a difference between the referenceimage 301 and the transformed image in step S1070.

According to the first embodiment, inspection processing correspondingto a region is performed for each of the picture region and theperipheral region on an inspection target medium. With this arrangement,even when a print position error occurs, a defect at the periphery of amedium can be detected more accurately.

Modification to First Embodiment

In the first embodiment, defect determination processing based on acomparison between the reference image 301 and the target image 302 isused for the picture region 305, and defect determination processingbased on a comparison between the target image 302 and featureinformation is used for the peripheral region 306. However, acombination of the inspection setting used for defect determination ofthe picture region 305 and the inspection setting used for defectdetermination of the peripheral region 306 is not limited to this. Forexample, the processing selection unit 203 may select defectdetermination processing based on a comparison between the referenceimage 301 and the target image 302 even for the peripheral region 306.In this case, the processing selection unit 203 can set, as thethreshold used for defect determination by the inspection processingunit 205, values different between inspection of the picture region 305and that of the peripheral region 306. For example, the inspectionprocessing unit 205 can detect a defect in the picture region 305 of aninspection target medium by comparing with the first threshold adifference between the transformed image 309 and the target image 302 inthe picture region 305. In contrast, the inspection processing unit 205can detect a defect in the peripheral region 306 of the inspectiontarget medium by comparing, with the second threshold different from thefirst threshold, a difference between the transformed image 309 and thetarget image 302 in the peripheral region 306. Even with thisarrangement, an inspection can be performed at inspection settingsdifferent between the picture region 305 and the peripheral region 306.

The processing selection unit 203 detects a defect in the picture region305 of an inspection target medium, whereas it can omit detection of adefect in the peripheral region 306 of the inspection target medium. Forexample, the inspection method may be set not to perform defectdetermination for the peripheral region 306 based on an input from theuser or a setting. Even with this arrangement, an inspection can beperformed at inspection settings different between the picture region305 and the peripheral region 306. Further, the processing selectionunit 203 may set the inspection method so as to perform defectdetermination using the same method between the picture region 305 andthe peripheral region 306 based on an input from the user or a setting.

Second Embodiment

In the second embodiment, similar to the first embodiment, thepresence/absence of a defect on an inspection target medium is inspectedusing inspection processing corresponding to each of the picture regionand the peripheral region. In the second embodiment, in particular, adefect is determined based on a comparison between a reference image anda target image also for the peripheral region of an inspection targetmedium. In the second embodiment, alignment processing corresponding toeach of the picture region and the peripheral region is performed todetermine a defect at inspection settings different between the pictureregion and the peripheral region. The functional arrangement of an imageprocessing apparatus 100 according to the second embodiment is similarto that of the image processing apparatus 100 according to the firstembodiment shown in FIG. 2 , and a difference from the first embodimentwill be described below.

A processing selection unit 203 selects inspection settings for apicture region 305 and a peripheral region 306. In the secondembodiment, the processing selection unit 203 selects the details ofalignment processing for inspection of the picture region 305, and thoseof alignment processing for inspection of the peripheral region 306.

An alignment unit 204 aligns a reference image 301 and a target image302 by the first method for inspection of the picture region 305. In thesecond embodiment, similar to the first embodiment, the alignment unit204 aligns the reference image 301 and the target image 302 based on thepicture region 305 for inspection of the picture region 305. Aninspection processing unit 205 compares the aligned reference image 301and the target image 302, detecting a defect in the picture region 305of the inspection target medium. The alignment unit 204 aligns thereference image 301 and the target image 302 by the second method forinspection of the peripheral region 306. In the second embodiment, thealignment unit 204 aligns the reference image 301 and the target image302 based on the periphery positions of the inspection target medium andreference medium for inspection of the peripheral region 306. Theinspection processing unit 205 compares the aligned reference image 301and the target image 302, detecting a defect in the peripheral region306 of the inspection target medium. Details of this processing will bedescribed below.

Processing performed by the image processing apparatus 100 according tothe second embodiment will be described with reference to the flowchartof FIG. 3 . Processes in steps S1010 to S1030 are similar to those inthe first embodiment. In step S1040, the processing selection unit 203selects the details of alignment processing for the picture region 305and those of alignment processing for the peripheral region 306 that areperformed in step S1050, as described above. In the second embodiment,similar to the first embodiment, alignment processing using affinetransformation based on reference points 307 a to 307 c is selected forthe picture region 305. For the peripheral region 306, alignmentprocessing using affine transformation based on the positions of fourvertices of a print medium is selected.

The reference image 301 and the target image 302 are aligned based onthe picture region 305 in step S1050 similarly to the first embodiment.That is, the alignment unit 204 can calculate the transformation X basedon the reference points 307 a to 307 c for the reference image 301, andgenerate a transformed image 309 by applying the transformation X to thereference image 301. In step S1050, the alignment unit 204 aligns thereference image 301 and the target image 302 based on the peripheralregion 306.

FIGS. 8A to 8C are views for explaining an outline of alignmentprocessing based on the peripheral region 306. FIGS. 8A and 8B show therelationship between the reference image 301, the target image 302, thepicture region 305, the peripheral region 306, and vertices 308 a to 308d of a print medium region 304. The alignment unit 204 aligns thereference image 301 and the target image 302 based on the coordinates ofthe vertices 308 a to 308 d of the respective print medium regions 304in the reference image 301 and the target image 302 that have beendetected in step S1030. For example, when geometric transformation isapplied to the reference image 301, the alignment unit 204 can calculatea geometric transformation for alignment so that the positions of thevertices 308 a to 308 d in the reference image 301 and the target image302 match with each other. In this embodiment, the alignment unit 204can calculate a transformation X′ of the peripheral region 306 from thereference image 301 to the target image 302 according to equation (3):A′X′=B′where

${A = \begin{pmatrix}{x1a} & {y1a} & 1 \\{x1b} & {y1b} & 1 \\{x1c} & {y1c} & 1 \\{x1d} & {y1d} & 1\end{pmatrix}}{B = \begin{pmatrix}{x2a} & {y2a} & 1 \\{x2b} & {y2b} & 1 \\{x2c} & {y2c} & 1 \\{x2d} & {y2d} & 1\end{pmatrix}}{X = \begin{pmatrix}{x11} & {x12} & {x13} \\{x21} & {x22} & {x23} \\0 & 0 & 1\end{pmatrix}}$Since A′ and B′ in equation (3) are not square matrices, the alignmentunit 204 calculates a Moore-Penrose pseudo-inverse matrix with respectto the matrix A′ and can calculate X′.

The alignment unit 204 generates a periphery transformed image 310 byapplying the calculated transformation X′ to each pixel in the printmedium region 304 of the reference image 301. FIG. 8C shows a state inwhich the target image 302 is aligned by alignment processing with theperiphery transformed image 310 obtained by transforming the referenceimage 301 using the transformation X′. At this stage, the positions ofthe vertices 308 a to 308 d of the print medium regions 304 and thepositions of the peripheral regions 306 in the periphery transformedimage 310 and the target image 302 match with each other. To thecontrary, the positions of the picture regions 305 in the peripherytransformed image 310 and the target image 302 may not match with eachother.

Finally, the alignment unit 204 generates an overall transformed image311 by compositing the periphery transformed image 310 and thetransformed image 309. FIGS. 9A to 9C are views for explaining theoverall transformed image 311. The alignment unit 204 generates theoverall transformed image 311 by combining an image corresponding to thepicture region 305 of the target image 302 out of the transformed image309 and an image corresponding to the position of the peripheral region306 of the target image 302 out of the periphery transformed image 310.Since the picture region 305 and peripheral region 306 of the targetimage 302 do not overlap each other, the alignment unit 204 can generatethe unique overall transformed image 311. The generated overalltransformed image 311 corresponds to the result of aligning the overallreference image 301 with the target image 302.

In this embodiment, processes in steps S1060 and S1070 can be performedsimultaneously. That is, the inspection processing unit 205 cancalculate a difference between the target image 302 and the overalltransformed image 311 for each pixel of the target image 302. Theinspection processing unit 205 can determine that a pixel having acalculated difference larger than a threshold is a defective pixel.Processing in step S1080 is similar to that in the first embodiment. Inthis fashion, the processing according to the second embodiment can beperformed.

Note that it is not essential to generate the overall transformed image311 in step S1050. For example, a defect may be detected using differenttransformed images for the picture region 305 and the peripheral region306. In this case, in step S1060, the inspection processing unit 205 cancalculate a difference between the periphery transformed image 310 andthe target image 302 for each pixel in the peripheral region 306, andcompare the difference with a threshold. In step S1070, the inspectionprocessing unit 205 can calculate a difference between the transformedimage 309 and the target image 302 for each pixel in the picture region305, and compare the difference with a threshold.

According to the second embodiment, inspection processing usingdifferent alignment processing corresponding to a region is performedfor each of the picture region and the peripheral region on aninspection target medium. With this arrangement, even when a printposition error occurs, a defect at the periphery of a medium can bedetected more accurately. Especially in the second embodiment, imagecomparison is performed even in the peripheral region, and a defect nearthe periphery of a medium can be detected more accurately.

Third Embodiment

In the first and second embodiments, a defect in a target image isdetected by comparing the target image with a reference image in atleast the picture region. Even in a conventional technique, a defect ina target image is detected by comparing the target image with areference image. In the first embodiment, a reference image is obtainedby reading a reference medium. In the third embodiment, a referenceimage is generated based on the reading results of reference media. Forexample, in the third embodiment, a reference image can be generatedbased on read images obtained by reading respective reference media.This arrangement increases the S/N ratio of the reference image, andhigh accuracy of defect inspection is expected.

If a print position error occurs on each of reference media, read imagesneed to be aligned before compositing them. If alignment is performed tomake pictures match with each other even in this case, similar to thefirst and second embodiments, a mismatch at the peripheries of the mediaincreases, and an improper reference image may be generated. Therefore,in the third embodiment, a picture region and a peripheral region on areference medium are determined, and a plurality of read images arecomposited into a reference image by compositing processingcorresponding to each region. The thus-generated reference image can beused to increase the accuracy of inspection according to the first andsecond embodiments. The reference image generated according to the thirdembodiment can also be used in inspection according to the conventionaltechnique.

FIG. 6 is a block diagram showing the functional arrangement of an imageprocessing apparatus 100 according to the third embodiment. Thefunctional arrangement of the image processing apparatus 100 accordingto the third embodiment is similar to that of the image processingapparatus 100 according to the first embodiment shown in FIG. 2 , and adifference from the first embodiment will be explained.

An image obtaining unit 201 obtains the first image serving a read imageof a reference medium representing a target print result, and the secondimage different from the first image. In this embodiment, an imageobtained by reading a reference medium at the first time by an imagereading device 105 will be called the first read image, and an imageobtained by reading another reference medium at the second or subsequenttime will be called the second read image. The image obtaining unit 201is similar to that in the first embodiment except that it obtains thefirst and second read images instead of a reference image 301 and atarget image 302. The image obtaining unit 201 can obtain a plurality ofsecond read images by reading respective reference images. The secondread images can be sequentially composited to the first read image. Thefirst read image or the first read image to which one or more secondread images are composited will sometimes be called a processed image.

A region determination unit 202, a processing selection unit 203, and analignment unit 204 are similar to those in the first embodiment exceptthat they process the processed image and the second read image insteadof the reference image 301 and the target image 302.

A compositing processing unit 206 composites the first and second readimages. The compositing processing unit 206 can generate a referenceimage that is compared with a read image of an inspection target mediumin order to detect a defect on the inspection target medium havingundergone printing. The compositing processing unit 206 performscompositing processing at compositing settings different between apicture region 305 and peripheral region 306 of the medium. In thisembodiment, the compositing processing unit 206 can generate a referenceimage by repeating compositing processing of the second read image to aprocessed image. An image obtained by compositing one second read imageto a processed image will be called a composite image.

In this embodiment, the compositing processing unit 206 generates animage in the picture region 305 of a composite image by compositing aprocessed image and the second read image. The compositing processingunit 206 generates an image in the picture region 305 of the compositeimage by compositing the processed image and the second read image inaccordance with alignment by the alignment unit 204. In this embodiment,the compositing processing unit 206 generates an image in the pictureregion 305 of the composite image by compositing the processed image andsecond read image aligned based on an object in the image.

The compositing processing unit 206 generates information used toinspect the peripheral region 306 on an inspection target medium at acompositing setting different from that of the picture region 305. Thecompositing processing unit 206 composites pieces of feature informationof images in the peripheral regions 306 of media out of the processedimage and the second read image. Such feature information can be used toinspect a peripheral region on an inspection target medium (target image302) in the first embodiment. In the third embodiment, the compositingprocessing unit 206 sets a pixel value in the peripheral region 306 of acomposite image at a feature value obtained by compositing the featurevalue of the peripheral region 306 of the processed image and that ofthe peripheral region 306 of the second read image. The obtainedcomposite image including the picture region 305 in which the alignedprocessed image and the second read image are composited, and theperipheral region 306 having a predetermined pixel value can be used asa reference image in the first and second embodiments.

Processing performed by the image processing apparatus 100 having theabove-described arrangement according to the embodiment will bedescribed below. FIG. 7 is a flowchart showing a sequence of processingperformed by the image processing apparatus 100. In the processing shownin FIG. 7 , a plurality of second read images are sequentiallycomposited to the first read image.

In step S1010, the image obtaining unit 201 obtains the first read imagesimilarly to the first embodiment except that the first read image isobtained instead of the reference image 301. The obtained first readimage is used as a processed image. In step S1020, the image obtainingunit obtains the second read image similarly to the first embodimentexcept that one of the second read images is obtained instead of thetarget image 302.

In step S1030, the region determination unit 202 detects the pictureregion 305 and the peripheral region 306 in the processed image byperforming region determination for the processed image similarly to thefirst embodiment. Similarly, the region determination unit 202 detectsthe picture region 305 and the peripheral region 306 in the second readimage by performing region determination for the second read image.Further, the region determination unit 202 calculates a feature valuefor the peripheral region 306 of the second read image. The featurevalue is used in step S1100. Only in the first loop, the regiondetermination unit 202 replaces, with the feature value of theperipheral region 306, the pixel values of all pixels belonging to theperipheral region 306 of the first read image (=processed image). As forthe feature value, the feature value of the peripheral region 306 inthis example is the average brightness of pixels belonging to theperipheral region 306, which will be described later.

In step S1040, the processing selection unit 203 selects the details ofcompositing processing to be performed for the picture region 305 instep S1100, and those of compositing processing to be performed for theperipheral region 306 in step S1110. In this embodiment, compositingprocessing based on averaging of the processed image and the second readimage is selected for the picture region 305, and compositing processingbased on the feature value is selected for the peripheral region 306. Adetailed processing selection method is similar to that in the firstembodiment.

In step S1050, the alignment unit 204 aligns the processed image and thesecond read image. In the third embodiment, similar to the firstembodiment, the processed image and the second read image are alignedbased on the picture region 305. For example, the alignment unit 204 cancalculate the transformation X of the picture region 305 from theprocessed image to the second read image based on reference points.Then, similar to the first embodiment, the alignment unit 204 generatesa transformed image by applying the calculated transformation X to eachpixel constituting a print medium region 304 in the processed image. Asa result, the alignment unit 204 can align the processed image and thesecond read image.

In step S1100, the inspection processing unit 205 performs for theperipheral region 306 compositing processing selected by the processingselection unit 203 in step S1040. As described above, in thisembodiment, the compositing processing unit 206 performs compositingprocessing based on the feature value for the peripheral region 306. Thefeature value in this embodiment is the representative brightness ofpixels belonging to the peripheral region 306. The significance of thefeature value may be similar to that in the first embodiment. In thisexample, the average brightness of pixels belonging to the peripheralregion 306 is used as the feature value.

For example, the compositing processing unit 206 performs compositingprocessing using the feature value for each pixel of the peripheralregion 306 in the second read image. More specifically, the compositingprocessing unit 206 calculates the average of a feature value (in thisexample, the average brightness of the peripheral region 306) in thesecond read image and a feature value in the processed image for eachpixel of the peripheral region 306.

As an example of the processing by the compositing processing unit 206,first, the compositing processing unit 206 can replace, with the featurevalue of the peripheral region 306, the pixel values of all pixelsbelonging to the peripheral region 306 of the second read image. In thisexample, the feature value is the average brightness of all pixelsbelonging to the peripheral region 306 of the second read image. Thecompositing processing unit 206 can calculate an average according toequation (4):I ₃(x, y)=(N×I ₁(x, y)+N×I ₂(x, y))/(N+1)   (4)where x and y are indices of each pixel in the peripheral region 306,I₁(x, y), I₂(x, y), and I₃(x, y) are pixel values in the processedimage, the second read image, and the composite image, respectively, andN is the number of the second read image to be composited. In the caseof processing the third, second read image, N=3.

As described above, the pixel value of the peripheral region 306 in thefirst read image is replaced with the average brightness of theperipheral region 306 of the first read image in step S1030. The pixelvalue of the peripheral region 306 in the second read image is alsoreplaced with the average brightness of the peripheral region 306 of thesecond read image in step S1100. Therefore, all pixels (x, y) belongingto the peripheral regions 306 of the first and second read images havethe same value. As a result, all the pixels of the peripheral region 306have the same value even in a composite image obtained from the firstread image (=processed image) and the second read image according toequation (4). The thus-obtained composite image is used as a processedimage in the next processing loop, so all the pixels of the peripheralregion 306 of the composite image have the same value even in subsequentprocessing loops. The I₃(x, y) value obtained from equation (4) is theaverage value (I₁(x, y)) of the processed image and the average value(I₂(x, y)) of the second read image, which are weighted by the number ofcomposited read images. In other words, the I₃(x, y) value is theaverage brightness of the peripheral regions 306 of the first and secondread images processed so far.

In step S1110, the compositing processing unit 206 performs for thepicture region 305 compositing processing selected by the processingselection unit 203 in step S1040. As described above, in thisembodiment, the compositing processing unit 206 performs compositingprocessing based on averaging of the processed image and the second readimage for the picture region 305. For example, the compositingprocessing unit 206 can composite the second read image and atransformed image obtained from the processed image by alignmentprocessing in step S1050 for each pixel of the picture region 305 in thesecond read image. More specifically, the compositing processing unit206 can calculate the pixel value of each pixel (x, y) of the pictureregion 305 of the composite image using equation (4) described above (inthis case, I₁(x, y) is the pixel value of the transformed image). Insteps S1100 and S1110, the composite image including the picture region305 and the peripheral region 306 is generated.

In step S1120, the compositing processing unit 206 determines whetherthere is a read image of a reference medium which has not beencomposited yet. When compositing a read image of the next referencemedium, the image obtaining unit 201 replaces the processed image withthe composite image. After that, the process returns to step S1020 tocomposite the processed image with the second read image of a newreference medium. In steps S1100 and S1110 in the next processing loop,the N value used in equation (4) is incremented by one.

If the compositing processing unit 206 determines in step S1120 thatread images of all reference media have been composited, the generatedcomposite image is output to an auxiliary storage device (not shown),and the processing in FIG. 7 ends. As described above, the outputcomposite image can be used as the reference image 301 in the first andsecond embodiments. By using the composite image generated in this wayas the reference image 301, high accuracy of inspection processing inthe first and second embodiments is expected.

In the above-described example, the compositing processing unit 206generates a composite image by sequentially calculating an averageaccording to equation (4) while repeating steps S1020 to S1120. However,the composite image generation method is not limited to this. Forexample, in step S1020, the image obtaining unit 201 can obtain aplurality of second read images as the reading results of a plurality ofreference media. In steps S1100 and S1110, the compositing processingunit 206 may composite all the first and second read images. Forexample, the compositing processing unit 206 can calculate an averageusing equation (5) instead of equation (4):I ₃(x, y)=(I ₁(x, y)+I₂₁(x, y)+I ₂₂(x, y)+ . . . +I _(2N−1)(x, y))/N  (5)where N is the total number of read images, I₁(x, y) is the pixel valueof the first read image, and I₂₁(x, y) to I_(2N−1)(x, y) are the pixelvalues of respective second read images. Even in this case, the pixelvalue of the peripheral region 306 of the composite image is the averageof the feature values of the peripheral regions 306 in the first andsecond read images. The pixel value of the picture region 305 of thecomposite image is the average pixel value of each pixel after the firstand second read images are aligned based on the picture of the pictureregion 305.

The compositing processing unit 206 may adopt the median of the pixelsof the first and second read images as the pixel value of the compositeimage. In this case, the compositing processing unit 206 can calculatethe pixel value of the composite image using equation (6) instead ofequation (5):I ₃(x, y)=median(I ₁(x, y), I ₂₁(x, y), I ₂₂(x, y), . . . , I _(2N−1)(x,y))   (6)where median is the function of calculating a median.

In this embodiment, the pixel value of the peripheral region 306 in aread image is replaced with the average brightness of the peripheralregion 306. Then, read images are composited according to equation (4)or the like, generating a composite image in which the peripheral region306 has a predetermined pixel value. However, the compositing processingunit 206 may generate feature information about the peripheral region306 of a reference image by another method. For example, the compositingprocessing unit 206 may calculate the average brightness of theperipheral regions 306 of the first and second read images and output itas feature information about the peripheral region 306 of the referenceimage. This feature information can be used as feature information aboutthe peripheral region 306 having a predetermined value that is comparedwith image information at each position of the peripheral region 306 ofthe target image 302 in the first embodiment. In this case, thecompositing processing unit 206 can output a reference image generatedby compositing a processed image and second read image aligned based onan object in the picture region 305, and feature information about theperipheral region 306 of this reference image. As another example, thecompositing processing unit 206 may generate a reference image bycompositing a processed image and second read image aligned based on anobject in the picture region 305, and replacing the pixel value of theperipheral region 306 of the obtained image with the featureinformation.

According to the third embodiment, a reference image is obtained bydetermining the picture region and peripheral region of a referencemedium, and compositing read images of a plurality of reference mediausing compositing processing corresponding to each region. With thisarrangement, even when a print position error occurs, a more properinspection reference image can be generated.

Fourth Embodiment

Also in the fourth embodiment, the picture region and peripheral regionof a reference medium are determined, and a plurality of read images arecomposited into a reference image by compositing processingcorresponding to each region. Especially in the fourth embodiment, thecompositing processing of a reference image is performed at compositingsettings different between the picture region and the peripheral regionby performing alignment processing corresponding to each of the pictureregion and the peripheral region. The functional arrangement of an imageprocessing apparatus 100 according to the fourth embodiment is similarto that of the image processing apparatus 100 according to the thirdembodiment shown in FIG. 6 , and a difference from the third embodimentwill be described below.

A processing selection unit 203 selects the details of compositingprocessing for a picture region 305 and a peripheral region 306. In thefourth embodiment, similar to the second embodiment, as the details ofalignment processing performed for the compositing processing of thepicture region 305, the processing selection unit 203 selects alignmentprocessing based on reference points in the picture region 305. Alsosimilar to the second embodiment, as the details of alignment processingperformed for the compositing processing of the peripheral region 306,the processing selection unit 203 selects alignment processing based onthe positions of four vertices of a print medium in the peripheralregion 306. The details of alignment processing performed by analignment unit 204 are similar to those in the second embodiment.

Even in the fourth embodiment, a compositing processing unit 206generates a composite image. Similar to the third embodiment, thepicture region 305 of the composite image is a composite image of aprocessed image and second read image aligned based on an object in theimage. The peripheral region 306 of the composite image is a compositeimage of a processed image and second read image aligned based on theperiphery position of the medium. The thus-obtained composite image canalso be used as a reference image in the first and second embodiments.

Processing performed by the image processing apparatus 100 having theabove-described arrangement according to the embodiment will bedescribed with reference to the flowchart of FIG. 7 . Processes in stepsS1010 to S1030 are similar to those in the third embodiment. Processingin step S1040 is also similar to that in the third embodiment exceptthat the processing selection unit 203 selects the details ofcompositing processing in the above-described manner.

In step S1050, the alignment unit 204 performs processing as in thesecond embodiment except that a processed image and the second readimage are aligned. By processing in step S1050, the transformation X′ ofthe peripheral region 306 from the processed image to the second readimage, and the transformation X of the picture region 305 from theprocessed image to the second read image are obtained. An overalltransformed image obtained by the processing in step S1050 has, in aregion corresponding to the position of the peripheral region 306 of thesecond read image, a peripheral transformed image obtained bytransforming the processed image using the transformation X′. Further,the overall transformed image has, in a region corresponding to theposition of the picture region 305 of the second read image, atransformed image obtained by transforming the processed image using thetransformation X.

In step S1100, the compositing processing unit 206 generates a compositeimage by compositing the overall transformed image obtained in stepS1050 and the second read image. In this embodiment, the compositingprocessing unit 206 can perform calculation according to equation (4)for each pixel of the print medium region 304 (that is, both the pictureregion 305 and the peripheral region 306) of the second read image. Inthis case, I₁(x, y) represents the pixel value of the overalltransformed image. In this fashion, a composite image having the pixelvalue I₃(x, y) calculated according to equation (4) can be generated.Processing in step S1120 is similar to that in the third embodiment.

According to the fourth embodiment, even when a print position erroroccurs, a more proper inspection reference image can be generated.Particularly in the fourth embodiment, image compositing processing isperformed even in the peripheral region, and a defect near the peripheryof a medium can be detected more accurately.

Fifth Embodiment

In the second embodiment, alignment processing based on the positions offour vertices of a print medium in the peripheral region 306 isperformed as an alignment method when performing inspection of theperipheral region 306. In the fifth embodiment, when comparing the pixelvalues of a target image and a reference image, pixels in apredetermined search range are compared as shown in FIG. 10B, details ofwhich will be described later. The inspection of a peripheral region 306can therefore be performed using images in inspection ranges cut outfrom the reference image and the target image to have a region of apredetermined number of pixels around a print medium region 304.

This arrangement can increase the defect detection accuracy when awrinkle or a distortion exists at the periphery of a print medium. Forexample, as shown in the enlarged view of FIG. 10A, a reference image2011 obtained by compositing read images of a plurality of referencemedia may have a dark portion generated by compositing paper white and ablack background at the periphery of the image. In addition, the blackbackground portion may be captured at a periphery corresponding to thepaper white portion of the print medium region 304 in a target image2020. According to the embodiment, when a black region or a dark regionis generated in the peripheral region of the reference image or thetarget image, detection of the region as a defect can be suppressed.

First, a method of generating a reference image by compositing readimages of a plurality of reference media in this embodiment will beexplained. According to the following method, a reference image can begenerated to include a background region at the periphery. However, itis not essential to use the reference image generated as follows forinspection of an inspection target medium.

Processing in the fifth embodiment is different from that in the fourthembodiment in processing of compositing the peripheral regions of aplurality of reference images. This processing in the fifth embodimentwill be described below with reference to the flowchart of FIG. 11 . Thefollowing processing can be used in combination with processes such asalignment and compositing processing of a picture region in the fourthembodiment. That is, a composite image can be generated by performingcompositing processing of a picture region 305 according to steps S1030to S1110 in FIG. 7 while compositing the peripheral region 306 and amargin region according to steps S1180 to S1210 in FIG. 11 .

In step S1170, an image obtaining unit 201 obtains the second read imageas in step S1020 of the fourth embodiment. In step S1180, a regiondetermination unit 202 extracts a margin region in addition to theperipheral region 306. The margin region is a region of a predeterminedpixel width provided on upper, lower, right, and left sides around theperipheral region 306 (or print medium region 304). Although the pixelwidth is not particularly limited, the predetermined pixel width is setto 20 pixels in the following description. Similar to theabove-described peripheral margin, a predetermined pixel width may beset based on a margin amount or print position error amount generatedwhen a printing apparatus 190 prints. For example, the margin region canbe extracted to include the black region of the background. If an errorof the print position of the printing apparatus with respect to theprint medium is about 10 pixels, the width of the margin region can beset to about 10 pixels. A mechanism for setting a predetermined pixelwidth by the user may be provided separately.

In step S1190, an alignment unit 204 aligns a processed image and thesecond read image based on the peripheral region 306 as in step S1050 ofthe fourth embodiment. That is, the alignment unit 204 can performalignment processing based on the positions of four vertices of a printmedium in the peripheral region 306. Similar to the fourth embodiment,the alignment unit 204 can transform the processed image so as to alignthe processed image with the second read image.

In step S1200, the alignment unit 204 obtains regions corresponding tothe peripheral region 306 and the margin region from the transformedprocessed image obtained in step S1190. The obtained image includesblack pixels corresponding to the background region around the printmedium region 304, like a reference image 2030 in FIG. 10B.

In step S1210, the compositing processing unit 206 composites the secondread image and the transformed processed image obtained in step S1190 inthe peripheral region 306 and the margin region. In this way, thecompositing processing unit 206 can generate the peripheral region 306and margin region of the composite image serving as a composite image ofthe processed image and second read image aligned based on the peripheryposition of the medium. The compositing processing unit 206 can generatea composite image by sequentially calculating an average according toequation (4) as in the fourth embodiment for both the peripheral region306 and the margin region. In the fifth embodiment, the compositingprocessing unit 206 may calculate a difference in pixel value betweenimages and if the difference exceeds a predetermined threshold, employthe pixel value of an image having a smaller pixel value withoutaveraging, in order to hold the contrast between the margin region andpaper white at the periphery of the image. Instead of calculating adifference in pixel value, the compositing processing unit 206 may adoptanother method of, for example, when the pixel value of the second readimage is smaller than a predetermined threshold, replacing it with apredetermined pixel value representing a black region.

As described above, similar to the fourth embodiment, the compositingprocessing unit 206 can generate the picture region 305 and marginregion of the composite image serving as a composite image of theprocessed image and second read image aligned based on the pictureregion 305. Accordingly, the composite image including the pictureregion 305, the peripheral region 306, and the margin region can begenerated.

Processing in step S1220 is similar to that in the fourth embodiment.After the end of compositing read images of all reference media, thegenerated composite image is output to an auxiliary storage device (notshown), and the processing ends. By this processing, a reference imagein which read images of reference media including the background regionat the periphery is obtained.

Next, processing in the fifth embodiment in which an inspection targetmedium is inspected by comparing a target image and a reference imageincluding the background region at the periphery will be explained withreference to the flowchart of FIG. 12 . In step S1230, the imageobtaining unit 201 obtains a reference image including the backgroundregion at the periphery. The reference image may be a composite imagegenerated according to the above-described method by compositing readimages of a plurality of reference media. In step S1240, the imageobtaining unit 201 obtains a target image as in step S1020 of the secondembodiment.

In step S1250, an inspection processing unit 205 extracts the pictureregion 305, the peripheral region 306, and the margin region from thetarget image. The method of extracting the picture region 305 and theperipheral region 306 is similar to that in the second embodiment. Themethod of extracting the margin region has been described above. Theinspection processing unit 205 can generate an image in which the marginregion is provided around the periphery of the print medium region 304.

In step S1260, similar to the second embodiment, the inspectionprocessing unit 205 aligns the reference image obtained in step S1230and the processed image obtained in step S1250. For example, theinspection processing unit 205 can transform the reference image into anoverall transformed image by performing alignment for the picture region305 and alignment for the peripheral region 306. As for the marginregion as well as the peripheral region 306, the inspection processingunit 205 can perform alignment based on the positions of four verticesof the print medium.

In step S1270, the inspection processing unit 205 decides a search rangeused to compare the pixel values of the reference image and the targetimage. In this embodiment, the margin region around the print mediumregion 304 is also extracted, and a detection error in the peripheralregion can be suppressed by comparing pixel values in a predeterminedrange. The size of the search range is not particularly limited and canbe 20 pixels, which is equal to the width of the margin region in thefollowing description. Alternatively, similar to the width of the marginregion, the size of the search range may be determined based on a marginamount or a print position error amount when the printing apparatus 190prints.

In step S1280, the inspection processing unit 205 calculates thedifference value of each value between the target image and thereference image in the peripheral region 306. As shown in FIG. 10B, theinspection processing unit 205 can calculate a difference value betweenthe pixel value of a pixel of interest of a reference image 2040 and thepixel value of each pixel of the reference image 2030 in the searchrange that corresponds to the pixel of interest. The search range may bea region whose vertical and horizontal distances from a pixelcorresponding to the pixel of interest in the reference image are equalto or shorter than the number of pixels decided in step S1270. In thisembodiment, the inspection processing unit 205 can calculate adifference value for each pixel by scanning the reference image in thevertical and horizontal directions by the number of pixels correspondingto the search range, and employ a minimum difference value as adifference used for defect determination.

In step S1290, the inspection processing unit 205 determines whether thecalculation of a difference value has ended for all the pixels of thetarget image. If the calculation has not ended, the process returns tostep S1280 to calculate a difference value for another pixel. If thecalculation has ended, the inspection processing unit 205 determinesbased on the difference value whether the pixel of the peripheral region306 of the target image is a defective pixel, similarly to the secondembodiment. Then, the inspection processing ends.

In the fifth embodiment, an inspection image having a margin regionaround a print medium region is used. When comparing the pixel values ofperipheral regions, a minimum value among the difference values ofpixels in a predetermined range is adopted as a difference value usedfor defect determination. For example, even when the pixel of interestof a target image represents paper white and a corresponding pixel of areference image represents the background, if a pixel of the same color(pixel representing paper white) as that of the pixel of interest existsnear the corresponding pixel of the reference image, the pixel is notdetermined as a defect. Even when the pixel of interest of a targetimage represents the background and a corresponding pixel of a referenceimage represents paper white, if a pixel of the same color (pixelrepresenting the background) as that of the pixel of interest existsnear the corresponding pixel of the reference image, the pixel is notdetermined as a defect. With this arrangement, a defect near theperiphery of a medium can be detected more accurately while suppressinga detection error.

Sixth Embodiment

In the above-described embodiments, inspection and alignment areperformed for each of the picture region and the peripheral region.However, inspection or alignment may be performed at an inspectionsetting or alignment setting for three or more regions. In the sixthembodiment, a method will be described in which a preprint region isnewly added to inspection target regions and an image on a print mediumis divided into three regions and processed. The preprint region is theregion of an image printed in advance on a print medium before actualprinting by a printing apparatus. Note that the preprint region mayexist at the center of the print medium.

Alignment processing and inspection processing for the preprint regionin the sixth embodiment, which is a difference from the secondembodiment, will be explained below with reference to the flowchart ofFIG. 13 . Alignment processing and inspection processing for the pictureregion and the peripheral region in the sixth embodiment are similar tothose in the second embodiment, and a description thereof will not berepeated.

In step S1300, as in step S1010 of the second embodiment, an imageobtaining unit 201 obtains a reference image serving as a read image ofa reference medium representing a target print result. In step S1310, asin step S1020 of the second embodiment, the image obtaining unit 201obtains a target image serving as a read image of an inspection targetmedium having undergone printing.

In step S1320, a region determination unit 202 performs regiondetermination on the target image. In this embodiment, the regiondetermination unit 202 extracts a picture region, a peripheral region,and a preprint region from the target image. The region determinationunit 202 can extract the picture region, the peripheral region, and thepreprint region by referring to a region map image. The region map imageis used to discriminate a preprint region and can be created in advanceby the user. The region map image is an image having the same verticaland horizontal widths as those of a processed image, and a differentpixel value is stored for each region. For example, the pixel of thepicture region may have a pixel value (R, G, B)=(255, 255, 255), that ofthe peripheral region may have a pixel value (R, G, B)=(0, 0, 0), andthe preprint region may have a pixel value (R, G, B)=(128, 128, 128).Needless to say, the pixel value of the pixel of each region is notlimited to this value. In this case, the region determination unit 202may extract a print medium region from the target image similarly to thesecond embodiment, and extract the picture region, the peripheralregion, and the preprint region from the print medium region based onthe region map image. Instead of using the region map image, the regiondetermination unit 202 may perform region determination based on featureinformation of an image similarly to the second embodiment.

In step S1330, a processing selection unit 203 selects an inspectionsetting for each region. In this embodiment, inspection is performed atthe inspection setting of the preprint region printed in advance on aninspection target medium and those of the print region and peripheralregion of the inspection target medium. For example, similar to thesecond embodiment, the processing selection unit 203 can selectalignment based on the picture region (that is, an object in the image)for the picture region, and alignment based on the periphery position ofthe medium for the peripheral region. As for the preprint region, theprocessing selection unit 203 can select alignment based on theperiphery position of the medium.

In step S1340, similar to the second embodiment, an alignment unit 204performs alignment processing at the inspection setting for theperipheral region. In step S1350, similar to the second embodiment, thealignment unit 204 performs alignment at the inspection setting for thepicture region.

In step S1360, the alignment unit 204 performs alignment at theinspection setting for the preprint region. In this embodiment,alignment based on the periphery position of the reference medium isperformed even for the preprint region as in step S1350. When an errorof the preprint position on a reference medium is smaller than an errorof alignment based on the feature point of a preprint picture, thealignment accuracy can be increased using the above-mentioned setting.When an error of the preprint position on a reference medium is largerthan an error of alignment based on the feature point of a preprintpicture, alignment based on the preprint region may be employed.

In step S1370, as in step S1050 of the second embodiment, the alignmentunit 204 generates an overall transformed image by combining the resultsof alignment processing on the picture region, peripheral region, andpreprint region of a reference image 301.

In step S1380, as in steps S1060 and S1070 of the second embodiment, aninspection processing unit 205 inspects a defect on the inspectiontarget medium by comparing the target image and the overall transformedimage. In this embodiment, defect detection in the preprint region usesthe reference image aligned based on the periphery position of themedium.

In step S1390, the inspection processing unit 205 determines whether theinspection has ended for all target images. If the inspection has notended, the process returns to step S1310. If the inspection has ended,the inspection processing ends.

According to the sixth embodiment, inspection can be performed even forthe preprint region at the inspection setting of the preprint region.For example, a defect in the preprint region can be detected accuratelyusing a reference image aligned based on the periphery position of amedium in inspection processing for the preprint region. The inspectionaccuracy can be increased not only by using inspection settingsdifferent between the picture region and the peripheral region, but alsoby using inspection settings different between the first region (pictureregion) of an image and the second region (preprint region) of theimage. An inspection setting different from those of the picture regionand the peripheral region may be used for the preprint region.

By using a similar method, a read image of a reference medium can alsobe composited at the compositing settings of the picture region, theperipheral region, and the preprint region. For example, a compositeimage used for inspection can be generated by performing compositingprocessing according to the fourth embodiment for the picture region andthe peripheral region, and performing alignment and then compositingprocessing in the above-described way for the preprint region. Forexample, compositing processing based on averaging may be selected forthe preprint region as well as the picture region.

Further Embodiments

The arrangements of the image obtaining unit 201 and the image readingdevice 105 are not particularly limited. For example, the type of imageobtained by the image obtaining unit 201 is not limited, and the imagemay be an RGB color image, a grayscale image, or a monochrome image.

Although the image reading device 105 reads a reference medium first andthen reads an inspection target medium in the first and secondembodiments, the generation order of a reference image and a targetimage by the image reading device is not particularly limited. In eachembodiment, the image obtaining unit 201 may obtain a reference image, atarget image, or a read image from an auxiliary storage device (notshown). The reference image and the target image may be arbitrary imagesthat allow calculating a difference between the reference image and thetarget image for inspection of the target image. The read image may bean arbitrary image that allows creating a reference image for inspectionof the target image.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™,a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2020-030849, filed Feb. 26, 2020, and Japanese Patent Application No.2020-064202, filed Mar. 31, 2020, which are hereby incorporated byreference herein in their entirety.

What is claimed is:
 1. An image processing apparatus comprising: one ormore memories storing instructions; and one or more processors thatexecute the instructions to: obtain a first image serving as a readimage of an inspection target medium having undergone printing, and asecond image serving as a read image of a reference medium representinga target print result; and detect a defect: in a print region of theinspection target medium by comparing the first image and the secondimage aligned based on an object in an image; and in a peripheral regionof the inspection target medium by comparing the first image and thesecond image aligned based on vertices of the inspection target mediumand the reference medium.
 2. The image processing apparatus according toclaim 1, wherein the one or more processors detect a defect in theperipheral region of the inspection target medium by comparing imageinformation at each position of the peripheral region of the first imagewith feature information having a predetermined value.
 3. The imageprocessing apparatus according to claim 2, wherein the image informationat each position of the peripheral region of the first image isbrightness of each pixel, and the feature information is averagebrightness of the peripheral region of the second image.
 4. The imageprocessing apparatus according to claim 1, wherein the one or moreprocessors detect a defect in the print region of the inspection targetmedium by comparing with a first threshold a difference between thefirst image and the second image in the print region, and detect adefect in the peripheral region of the inspection target medium bycomparing with a second threshold different from the first threshold adifference between the first image and the second image in theperipheral region.
 5. An image processing apparatus comprising: one ormore memories storing instructions; and one or more processors thatexecute the instructions to: obtain a first image serving as a readimage of a medium having undergone printing, and a second imagedifferent from the first image; and composite the first image and thesecond image by performing compositing processing at compositingsettings different between a print region and peripheral region of themedium.
 6. The image processing apparatus according to claim 5, whereinthe one or more processors composite pieces of feature information ofimages in the peripheral region of the medium out of the first image andthe second image.
 7. The image processing apparatus according to claim5, wherein the one or more processors execute the instructions togenerate a composite image of the first image and the second image forthe peripheral region by compositing the first image and the secondimage aligned based on a periphery position of the medium.
 8. The imageprocessing apparatus according to claim 5, wherein one or moreprocessors execute the instructions to generate a composite image of thefirst image and the second image for the print region by compositing thefirst image and the second image aligned based on an object in an image.9. The image processing apparatus according to claim 1, wherein: the oneor more processors execute the instructions to determine the printregion and the peripheral region in the first image or the second image,the determined peripheral region is a region within a predetermineddistance from an outline of the medium in the first image or the secondimage, and the predetermined distance is determined based on one of amargin amount or a print position error amount when a printing apparatusprints on the medium in the first image or the second image.
 10. Theimage processing apparatus according to claim 1, wherein the one or moreprocessors execute the instructions to: determine the print region andthe peripheral region in the first image or the second image; and detecta marker added to the medium in the first image or the second image, anddetermine the peripheral region based on coordinates of the marker. 11.The image processing apparatus according to claim 1, wherein: theinspection target medium has a region where printing has been performedin advance, and the one or more processors execute the instructions toperform inspection at respective inspection settings for the regionwhere printing has been performed in advance, the print region, and theperipheral region of the inspection target medium.
 12. The imageprocessing apparatus according to claim 11, wherein the one or moreprocessors detect a defect in the region where printing has beenperformed in advance on the inspection target medium by comparing thefirst image and the second image aligned based on a periphery positionof the inspection target medium.
 13. An image processing methodcomprising: obtaining a first image serving as a read image of aninspection target medium having undergone printing, and a second imageserving as a read image of a reference medium representing a targetprint result; detecting a defect in a print region of the inspectiontarget medium by comparing the first image and the second image alignedbased on an object in an image; and detecting a defect in a peripheralregion of the inspection target medium by comparing the first image andthe second image aligned based on vertices of the inspection targetmedium and the reference medium.
 14. An image processing methodcomprising: obtaining a first image serving as a read image of a mediumhaving undergone printing, and a second image different from the firstimage; and compositing the first image and the second image byperforming compositing processing at compositing settings differentbetween a print region and peripheral region of the medium.
 15. Anon-transitory computer-readable medium storing a program executable bya computer execute a method comprising: obtaining a first image servingas a read image of an inspection target medium having undergoneprinting, and a second image serving as a read image of a referencemedium representing a target print result; detecting a defect in a printregion of the inspection target medium by comparing the first image andthe second image aligned based on an object in an image; and detecting adefect in a peripheral region of the inspection target medium bycomparing the first image and the second image aligned based on verticesof the inspection target medium and the reference medium.
 16. Anon-transitory computer-readable medium storing a program executable bya computer to perform a method comprising: obtaining a first imageserving as a read image of a medium having undergone printing, and asecond image different from the first image; and compositing the firstimage and the second image by performing compositing processing atcompositing settings different between a print region and peripheralregion of the medium.